In background subtraction, it is challenging to detect foreground objects in the presence of complex background motions including waving trees, rippling water, illumination changes, etc. In order to solve this problem, a codebook-based object detection algorithm is proposed in the paper. Given that in actual scene the change of background reflects on brightness, color space is transformed from RGB space to YUV space for video sequences. Then the algorithm establishes a Box model which makes the codewords representation and training period more compact than the standard codebook. Besides, A local updated method, namely through frame difference to detect the region of variation, is incorporated into layered codebook to update the background real-timely, thus achieving more accurate foreground detection. Comparative results indicate that the algorithm can handle scenes containing moving backgrounds or illumination variations, and it achieves robust object detection for different types of video.